The Wiley Handbook of Cognition and Assessment by Andre A. Rupp Jacqueline P. Leighton & Jacqueline P. Leighton

Author:Andre A. Rupp,Jacqueline P. Leighton & Jacqueline P. Leighton
Language: eng
Format: epub
ISBN: 9781118956618
Publisher: John Wiley & Sons, Inc.
Published: 2016-11-14T00:00:00+00:00

13

Diagnostic Classification Models

Laine Bradshaw

Assessment is a complex endeavor that requires not only capturing, but also reliably describing, target components of the mind’s knowledge web and reasoning facilities. Recent demands in education further intensify the challenges in assessment: Users in both operational testing and research arenas seek assessments that yield more nuanced descriptions of knowledge and reasoning than typically are provided by educational assessments. Large‐scale tests have been and continue to be critiqued because they are not designed to offer reliable, detailed feedback about what students do and do not understand (e.g., Mislevy, Almond, & Lukas, 2004; Perie, Marion, Gong, & Wurtzel, 2007; Snow & Lohman, 1989). Moreover, the detection of students’ strengths and weaknesses in reasoning is increasingly viewed as an essential aspect of an effective educational assessment system (Center for K‐12 Assessment and Performance Management, 2014; Huff & Goodman, 2007).

Providing this type of detailed feedback in operational settings is inextricably tied to research. Research‐based theories of cognition and learning are required to form the basis of more intricate assessment designs (Nichols, 1994). In turn, assessments, along with psychometric models that make explicit the connection between the learning theory and assessment design, are needed to test posited theories on a large scale. Educational researchers can use assessments as research tools to gather empirical evidence for falsifying detailed hypotheses of knowledge as an integral part of refining and building robust theories (Rupp, Templin, & Henson, 2010; Templin, Bradshaw, & Paek, in press). Using an iterative process, refined theories lead to strengthened assessment designs, and strengthened designs lead to stronger empirical evidence to further hone theory.

Creating theory‐based assessments to fulfill the needs of today’s assessment systems and research requires the coordination of three acute understandings: (1) learning theories that clearly define latent constructs and their constituent components, (2) task construction that adequately elicits observable responses as manifestations of construct components, and (3) psychometric models that aptly characterize the response–construct relationships. As learning theories advance and become better able to tease apart nuances of an overarching construct, distinct components of the construct can emerge as separable traits of interest to be studied and assessed. Modeling multiple traits on a single test, together viewed as a multidimensional construct, necessitates the use of a multidimensional psychometric model. Holding other factors constant, multidimensional psychometric models require more complex estimation algorithms that necessitate longer tests and/or larger samples to yield reliable descriptions of each trait. Thus, a tension exists between gaining more detailed information from an assessment and keeping assessment conditions (e.g., test length) feasible within the practical constraints of a given context.

Diagnostic classification models (DCMs; Rupp & Templin, 2008a; Rupp et al., 2010) are a class of psychometric models that hold promise for reliably estimating high dimensionality under feasible testing conditions in order to provide a multivariate view of students’ strengths and weaknesses (Templin & Bradshaw, 2013). Estimation efficiency is a result of the way DCMs characterize each dimension. Namely, each dimension is an ordinal trait such that DCM examinee estimates are probabilistic categorizations of examinees according to levels of the trait.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.